This invention relates generally to systems and methods for facilitating playing music. More particularly, the invention relates to systems, apparatus and methods for providing a graphical animation for use in playing musical instruments, particularly keyboard instruments.
The invention relates generally to automated multimedia systems, and, more specifically, to a computer-based system for prompting the player of a musical keyboard.
Playing a musical keyboard can be an enjoyable experience. Most people have attempted it, but a relative few have mastered it, largely because of the initial difficulty and the hours of dedicated study and practice required. The many competing claims on a person""s time usually leave insufficient room for the regimen of keyboard practice. Parents who spend thousands of dollars on piano lessens for their children, often see their youngsters quit in frustration after a couple of years of struggle.
Various schemes, often based on computers, have been devised to assist with the learning process. Some of this software is very good at providing instruction in traditional keyboard methods in which the student learns to read the notes on the staff of a musical score, whether it be printed or displayed on a computer monitor. For example, Yamaha Corporation produces xe2x80x9cDisklavier Pro 2000xe2x80x9d including an LCD screen to view digital sheet music and video music performances that sync to the moving keys.
A major obstacle remaining for many students of the traditional method, however, is the large amount of continuous information processing required of the brain to translate the arcane notation of printed music into the corresponding motor commands to the fingers. Keeping track of sharps and flats amid a stream of chromatic signs, recognizing notes well above or below the staff, and executing of all the various notational nuances can be particularly daunting.
Some schemes attempt to address this problem by using a system of lights near or inside of the keys to indicate which keys are to be played at a given time, but this approach can only crudely guide the performance. The lights are unable to convey much of the auxiliary information coded in a piano score. There are about a dozen primary parameters that can be associated with a note to carry instructions as to how that note should be played. A key""s light being either on or off can transmit only a fraction of this information. Another serious shortcoming of these methods is the inability of the user to scan upstream in the flow of approaching notes. This deficiency impairs his ability to smoothly position his hands in anticipation of notes drawing near their instant of play. Prior art with relevance to the present invention includes:
U.S. Pat. No. 4,926,734 issued to Rickey sets forth GRAPHIC/TACTILE MUSICAL KEYBOARD AND NOMOGRAPHIC MUSIC NOTATION in which a graphic/tactile musical instruction system includes a three-row equal temperament whole tone graphic/tactile keyboard and a nomographic system of music notation. The nomographic notation is correlated to the keyboard by nomographically marking the lines and spaces of the stave adjacent the key signature and similarly graphically marking the notes played on the upper or lower row with a diagonal slant.
U.S. Pat. No. 5,153,829 issued to Furuya, et al. sets forth MULTIFUNCTION MUSICAL INFORMATION PROCESSING APPARATUS which includes: a unit for displaying on a screen a musical score, keyboard and tone time information to be inputted; a unit for designating the position of the keyboard, and tone time information, respectively displayed on the display unit; a unit for storing musical information produced through designation by the designating unit of the position of the keyboard and tone time information displayed on the display unit; a unit for controlling the display of the musical score, keyboard and tone time information on the screen of the display unit, and for controlling the display of a pattern of musical tone or rest on the musical score on the display unit in accordance with the position of the keyboard and tone time information respectively designated by the designating unit; and a unit for generating a musical tone by reading the musical information stored in the storage unit.
U.S. Pat. No. 5,315,911 issued to Ochi sets forth MUSIC SCORE DISPLAY DEVICE including a storage device for storing music score data, a display device for displaying the music score data for each specified section, and a comparator for comparing the musical playing data supplied from a player with the music score data. The music score display device further has a retrieval device for retrieving a playing position according to the compared result of the comparator, and a display controller for controlling display section of the display device based on the retrieval result of the retrieval device.
U.S. Pat. Nos. 5,533,903 and 5,746,605 issued to Kennedy set forth METHOD AND SYSTEM FOR MUSIC TRAINING including a method of and computer system for training a student to play a musical work having an identifiable rhythm structure, the musical work being decomposable into a plurality of variations each maintaining the identifiable rhythm structure. According to the method, a musical score of each variation is generated and then the musical scores are arranged in order of rhythmic difficulty. An animated movie is then generated, the movie comprising the music score of at least one variation, a sound recording of the music score, a graphic device that highlights the identifiable rhythm structure and a representation of how the music score would be played on a musical instrument. The animated movie is then presented to the student.
U.S. Pat. No. 5,583,308 issued to Owen sets forth MUSICAL EFFECTS APPARATUS AND TONE CONTROL PROCESS FOR A MUSICAL INSTRUMENT. In an automated system for learning to play a musical instrument, music data and tone control data for a musical composition of an exemplary musical instrument is stored in machine readable form. An audio signal generated by a musical instrument being played, corresponding to the exemplary musical instrument, is inputted through a musical effects apparatus wherein the musical tone of the musical instrument being played is mixed with the stored music data. The musical effects apparatus is utilized, under digital control, to match the musical tone of the musical instrument being played with that of the exemplary musical instrument.
U.S. Pat. No. 5,585,583 issued to Owen sets forth INTERACTIVE MUSICAL INSTRUMENT INSTRUCTION SYSTEM in which a musical instrument instruction system utilizes a CD-ROM electronic storage medium to store audio and video musical instrument instructional materials. Instructional materials are accessed utilizing a computer which provides an interactive system in which animation, voice, music, video and imagery are integrated in essentially real time and under complete control of the music student. The system permits the student to access the instructional materials in a variety of ways, and control the parameters governing the presentation of the instructional materials, including control of the tempo of the music and whether or not displays of the maimer of playing the musical instrument will be shown with a displayed notation or tablature relating to the selected musical piece. A mixer is provided for combining musical output of the practice instrument with musical output generated by the computer corresponding to the selected musical piece.
U.S. Pat. No. 5,690,496 issued to Kennedy sets forth MULTIMEDIA PRODUCT FOR USE IN A COMPUTER FOR MUSIC INSTRUCTION AND USE in which a computer is used to assist a student to learn to play a given musical work previously recorded on a master recording. A method of making the product is described which ensures that multimedia representations of the work are properly synchronized to the actual tempo variations, if any, in the master recording. A method of using the product for music training is also described.
U.S. Pat. No. 5,728,960 issued to Sitrick sets forth MULTI-DIMENSIONAL TRANSFORMATION SYSTEMS AND DISPLAY COMMUNICATION ARCHITECTURE FOR MUSICAL COMPOSITIONS in which a musical display system produces a display presentation in one of a plurality of formats from an original musical composition, the formats being created from the original musical composition responsive to input variables and a selected operating mode. The system is comprised of means to receive the original composition, a memory, a selection subsystem, a controller, and a display subsystem. The display system provides for selection of original compositions, creation of derivative compositions, distribution of compositions, monitoring of each performer""s performance, and also allows for retrieval and editing of musical compositions, such as changing keys, pitch, tempo, and many other parameters.
U.S. Pat. No. 5,908,997 issued to Arnold, et al. sets forth ELECTRONIC MUSIC INSTRUMENT SYSTEM WITH MUSICAL KEYBOARD comprising: an electronic music instrument, having selectable groups of reproducible sounds and individually selectable reproducible sounds; a signal generator for energizing an audio amplifier responsive to different digital audio sources, including the groups of sounds and the individual sounds a graphical user interface for displaying at least one control graphic representing controllable parameters of the audio signals generated by the generator; and, a controller responsive to operation of the control graphic for adjusting the controllable parameters of the generator and for selectively coupling different ones of the sources to the generator. The graphical user interface can comprise: a video display; and, a touch-responsive overlay. The controllable parameters can include multiple instrument sound selection and sound layer assignment, controlled responsive to operation of the at least one control graphic.
U.S. Pat. No. 6,042,555 issued to Kramer, et al. sets forth FORCE-FEEDBACK INTERFACE DEVICE FOR THE HAND in which a man-machine interface is disclosed which provides force information to sensing body parts. The interface is comprised of a force-generating device (106) that produces a force which is transmitted to a force-applying device via force-transmitting means. For example a glove employing force feedback may be programmed to teach finger dexterity, finger timing and even the motions necessary to learn some musical instruments. If the user were learning the piano, as fingers are flexed, the user would receive fingertip pressure from virtual keys signifying to the user that he had pressed the key.
U.S. Pat. No. 6,066,791 issued to Renard, et al. sets forth SYSTEM FOR INSTRUCTING THE PLAYING OF A MUSICAL INSTRUMENT which discloses a novel method and apparatus for instructing a student how to play a musical instrument. The method includes providing a display device, displaying a staff on the display device, displaying an image on the display device, and instructing the student to focus on the image while preferably using a musical instrument to play the notes on the staff, wherein the image moves generally in the direction of the progression of notes on the staff while the student attempts to play the, notes with the musical instrument. As the image moves, the image preferably moves in a direction generally parallel to the staff.
Although the prior art addresses musical keyboard playing in various ways, none includes a live video image of the player""s hands interacting with on-screen animation. For those who would like to read keyboard music and play it with their own hands, but who would welcome a simpler alternative to mastering the skill of sight reading a musical score, there is a need for a system that can reduce the amount of mental information processing involved, while preserving all of the information a score contains. The present invention fulfills this need, and provides other distinct advantages.
The present invention provides a method of representing the content of a musical score in an animated format that graphically prompts the user of a keyboard instrument. Intended primarily for pleasure, the graphic format elucidates features of the score in a way that may also be useful as a supplement to conventional keyboard instruction. The animation is displayed on the screen of a computer monitor together with a live video image of the user""s hands on a musical keyboard. Viewing only the images on the monitor, the user guides his fingers to the keys targeted by the animation. He is thus able to play the proper keys on the real keyboard without diverting his eyes from the monitor. At the same time, he bypasses the intense mental information processing involved in reading printed sheet music.
In the preferred embodiment the monitor is supported at eye level near the position otherwise occupied by sheet music. A video camera is mounted about four feet above the center of the keyboard, and is directed downward. The resulting keyboard image is displayed at the bottom and across the full width of the monitor screen. Across the top of the screen is a tool bar for controlling and monitoring the displayed material. The mid-screen area is devoted to the animation. Here the horizontal co-ordinate represents position along the keyboard in one-to-one correspondence with the bordering keyboard image below. The vertical co-ordinate represents time, increasing from bottom to top. Objects higher on the screen represent events that occur later than objects lower on the screen. The present time is defined as the lower edge of the animation area where it borders the keyboard image. All objects in the animation area move steadily downward until they touch the border, at which point they have arrived at the present.
A musical note is represented by a xe2x80x9cspritexe2x80x9d typically elliptical in shape, initially appearing directly above, and moving down-screen toward the image of the corresponding key. The distance of the sprite from the key image is a measure of the time remaining until the key is to be played. This shrinking distance gives the user time to take aim at the targeted key, and to anticipate the moment of play.
To keep the screen uncluttered, a sprite does not appear until the corresponding note is within one beat of play. The vertical length of the sprite represents the duration of the note. A quarter-note sprite, for example, is twice as long as an eighth-note sprite. The key is played when the sprite first touches the key image. While maintaining the same downward speed, the sprite then appears to slide behind the key image, and the key is held until the last of the sprite disappears from view.
Through the use of color, shape, and other features, the sprites are designed to carry all of the information contained in the musical score. For example blue sprites represent notes to be played with the left hand, and green with the right hand. Both colors switch to red at the instant of contact with the key image to announce the moment of play. The degree of blue or green color saturation indicates the loudness of a note, ranging from pale colors for pianissimo, to rich colors for fortissimo. Pedaling is represented by a black bottom edge on a sprite for pedal down, and a black upper edge for pedal up.
The style of attack is indicated by the degree to which the leading and trailing edges of the elliptical sprite are clipped, ranging from no clipping for the smoothly joined notes of legato, through progressively more severe clipping for portamento and staccato. A totally squared off leading edge indicates the xe2x80x9chammeredxe2x80x9d note requested by a martellato.
Fingering is suggested for each note by an Arabic numeral within a small circle of the same color as the approaching sprite, and fixed in position with its lowest point in contact with the targeted key. The colored circle helps to identify the key while the sprite is still some distance away.
The circle appears simultaneously with the sprite, and disappears when touched by the sprite.
The various parameters for a note are included as fields of a record in a xe2x80x9cpiece filexe2x80x9d, which is an array of such records for all of the notes in a particular piece. The first record in the array is a dummy note with its fields containing initialization information such as the time signature, default metronome setting, and total number of measures in the piece. A library of piece files is made available for downloading from the Internet, on CD-ROM or other storage medium for loading into computer memory.
The user exercises control over the animation from two access points. The first is the Set-Up Screen that appears just after an Opening Screen when the program is launched. The Set-Up Screen is where the user chooses a piece from the library, and initializes the fundamental operating parameters, such as the metronome setting, start and stop points, the magnitude of tempo variations, and whether to repeat the selected measures when finished. There he also selects from a series of options for audio features. For some pieces there will be a choice of the style of different artists, as reflected in the tempo variations and treatment of loud and soft passages. The user can also return to the Set-Up screen at any time to change these settings. The second control point is the monitoring and control tool bar at the top of the Action Screen where the animation is displayed. These controls let the user jump backward or forward in the piece, start, stop, or pause the action, etc.
The invention does not magically eliminate the need for practice. One still must play a piece many times to become proficient, but the mental effort is greatly reduced. Freed of the burden of reading the sheet music, the user can concentrate on training his fingers to quickly assume the indicated patterns. The user""s interaction with the animation is rather like that of a video game, and practice is fun. Because it is fun, it happens, and rapid progress is the result.
The accompanying drawings and detailed description offer further clarification of the features and advantages of the invention.